{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "45d5b217-0233-4fae-9693-3a5cf741d82b",
   "metadata": {},
   "source": [
    "物联网与大数据第四章笔记\n",
    "姚龙飞"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "def1651f-b0c2-43f7-a9f3-336bd660e7fd",
   "metadata": {},
   "source": [
    "# 第四章 分类：基本概念、决策树与模型评估"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c953ec9-47a9-4cd4-9e5e-da1ee3fc6d29",
   "metadata": {},
   "source": [
    "## 4.1 预备知识\n",
    "\n",
    "### 4.1.1 分类任务的定义\n",
    "- 分类任务是确定对象属于哪个预定义的目标类的问题。\n",
    "- 输入数据是记录集合，每条记录称为实例或样例，表示为元组  $ (x, y) $ ，其中  $ x $  是属性集合， $ y $  是类标号。\n",
    "- 类标号必须是离散属性，这是分类与回归任务的关键区别。\n",
    "\n",
    "### 4.1.2 分类模型\n",
    "- 分类模型（classification model）或目标函数（target function）是将属性集  $ x $  映射到预定义类标号的函数。\n",
    "- 分类模型可用于描述性建模和预测性建模。\n",
    "\n",
    "### 4.1.3 分类与回归的区别\n",
    "- 回归任务中目标属性是连续的，而分类任务中目标属性是离散的。\n",
    "\n",
    "## 4.2 解决分类问题的一般方法\n",
    "\n",
    "### 4.2.1 训练集和检验集\n",
    "- 使用训练集（training set）建立分类模型，该模型应用于检验集（test set）进行评估。\n",
    "- 检验集包含未知类标号的记录。\n",
    "\n",
    "### 4.2.2 混淆矩阵\n",
    "- 混淆矩阵用于评估分类模型性能，包含实际类标号与预测类标号的对比。\n",
    "\n",
    "### 4.2.3 性能度量\n",
    "- 准确率（accuracy）和错误率（error rate）是常用的性能度量。\n",
    "\n",
    "## 4.3 决策树归纳\n",
    "\n",
    "### 4.3.1 决策树的工作原理\n",
    "- 决策树通过一系列问题将数据集划分为更纯的子集。\n",
    "- 决策树由结点和有向边组成，包括根结点、内部结点和叶结点。\n",
    "\n",
    "### 4.3.2 如何建立决策树\n",
    "- Hunt算法是基础的决策树算法，通过递归划分训练记录建立决策树。\n",
    "\n",
    "### 4.3.3 表示属性测试条件的方法\n",
    "- 属性测试条件的表示方法取决于属性类型，包括二元属性、标称属性、序数属性和连续属性。\n",
    "\n",
    "### 4.3.4 选择最佳划分的度量\n",
    "- 不纯性度量（impurity measures）用于确定划分记录的最佳方法，包括熵（Entropy）、Gini指标和分类错误（Classification error）。\n",
    "\n",
    "### 4.3.5 增益率\n",
    "- 增益率（gain ratio）是考虑属性测试条件产生的输出数的划分标准。\n",
    "\n",
    "### 4.3.6 决策树归纳算法\n",
    "- TreeGrowth算法框架用于构建决策树，包括创建新结点、找到最佳分裂、分类叶结点和停止条件。\n",
    "\n",
    "### 4.3.7 决策树归纳的特点\n",
    "- 决策树归纳是一种非参数方法，易于解释，对噪声具有鲁棒性。\n",
    "\n",
    "## 4.4 模型的过分拟合\n",
    "\n",
    "### 4.4.1 过分拟合与拟合不足\n",
    "- 分类模型的误差分为训练误差和泛化误差。\n",
    "- 过分拟合是指模型在训练集上表现良好，但在未知记录上性能差。\n",
    "\n",
    "### 4.4.2 噪声导致的过分拟合\n",
    "- 训练数据中的噪声可能导致模型过分拟合。\n",
    "\n",
    "### 4.4.3 缺乏代表性样本导致的过分拟合\n",
    "- 训练集中缺乏代表性样本可能导致过分拟合。\n",
    "\n",
    "### 4.4.4 泛化误差估计\n",
    "- 使用再代入估计、结合模型复杂度、估计统计上界和使用确认集来估计泛化误差。\n",
    "\n",
    "### 4.4.5 处理决策树归纳中的过分拟合\n",
    "- 先剪枝和后剪枝是避免过分拟合的策略。\n",
    "\n",
    "## 4.5 评估分类器的性能\n",
    "\n",
    "### 4.5.1 保持方法\n",
    "- 将数据集划分为训练集和检验集，评估模型在检验集上的性能。\n",
    "\n",
    "### 4.5.2 随机二次抽样\n",
    "- 多次重复保持方法以改进性能估计。\n",
    "\n",
    "### 4.5.3 交叉验证\n",
    "- 每个记录用于训练和检验的次数相同，提供无偏估计。\n",
    "\n",
    "### 4.5.4 自助法\n",
    "- 有放回抽样训练记录，用于估计模型性能。\n",
    "\n",
    "## 4.6 比较分类器的方法\n",
    "\n",
    "### 4.6.1 估计准确度的置信区间\n",
    "- 使用二项式实验建模来推导置信区间。\n",
    "\n",
    "### 4.6.2 比较两个模型的性能\n",
    "- 使用正态分布近似来比较两个模型的错误率差异。\n",
    "\n",
    "### 4.6.3 比较两种分类法的性能\n",
    "- 使用k折交叉验证比较两种分类法的性能。\n",
    "\n",
    "\n",
    "\n",
    "### 混淆矩阵（表4-2）\n",
    "\n",
    " $$\n",
    "\\begin{array}{c|cc}\n",
    " & \\text{预测的类=1} & \\text{预测的类=0} \\\\\n",
    "\\hline\n",
    "\\text{实际的类=1} & f_{11} & f_{10} \\\\\n",
    "\\text{实际的类=0} & f_{01} & f_{00} \\\\\n",
    "\\end{array}\n",
    "$$ \n",
    "## 重要函数公式\n",
    "\n",
    "### 准确率（Accuracy）\n",
    "\n",
    " $$ \\text{准确率} = \\frac{f_{00} + f_{11}}{f_{00} + f_{01} + f_{10} + f_{11}} $$ \n",
    "\n",
    "### 错误率（Error Rate）\n",
    "\n",
    " $$ \\text{错误率} = \\frac{f_{01} + f_{10}}{f_{00} + f_{01} + f_{10} + f_{11}} $$ \n",
    "\n",
    "### 熵（Entropy）\n",
    "\n",
    " $$ \\text{Entropy}(t) = -\\sum_{i=1}^{c} p(i|t) \\log_2 p(i|t) $$ \n",
    "\n",
    "### 分类错误（Classification Error）\n",
    "\n",
    " $$ \\text{Classification Error}(t) = 1 - \\max_{i}[ p(i|t) ] $$ \n",
    "\n",
    "### 增益率（Gain Ratio）\n",
    "\n",
    " $$ \\text{Gain Ratio} = \\frac{\\text{Gain}(D, A)}{\\text{Split Info}(D, A)} $$ \n",
    "\n",
    "### 置信区间（Confidence Interval）\n",
    "\n",
    " $$ \\text{置信区间} = \\left( \\text{acc} - z_{\\alpha/2} \\sqrt{\\frac{\\text{acc}(1-\\text{acc})}{N}}, \\text{acc} + z_{\\alpha/2} \\sqrt{\\frac{\\text{acc}(1-\\text{acc})}{N}} \\right) $$ "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
